OCTA Repeatability & Reliability — Clean Data
Jae Lee, Bruce Burkemper, Dom Grisafe, Grace Richter
2019-11-26
Data
This data was taken from a clean dataset that Jae emailed out on November 26th, 2019.
Jae organized the retina by six different methods. He created OCTA variable scores for the overall image, 2 hemifields, 4 quadrants, 6 sectors, 12 clock hours, and an annular ring.
Annular image scoring was also assessed, and ICC’s were similar, but slightly lower than global. Results for annular image scoring are not shown. For now, only the overall “global” image scoring will be shown.
Zeiss commercial vessel area density and flux variables are also shown for the global image scoring.
Data Wrangling
Rename Variables
This section will rename the variables to match the dictionary
Label Variables
This section will label variables appropriately.
Data Dictionary
| Variable | Label |
|---|---|
| ptid | Patient Identification |
| Sociodemographic Variables | |
| age | Age |
| female | Sex |
| Systemic Clinical Variables | |
| htn_dx | Hypertension Diagnosis |
| dm_dx | Diabetes Mellitus Diagnosis |
| nro_dx | Optic Neuropathy |
| Ophthalmic Clinical Variables | |
| dr_dx | Diabetic Retinopathy Diagnosis, Ordinal |
| dr_dx_bin | Diabetic Retinopathy Diagnosis, Binary |
| glc_sus_dx | Glaucoma Suspect Diagnosis |
| glc_dx | Glaucoma Diagnosis, Ordinal |
| glc_dx_bin | Glaucoma Diagnosis, Binary |
| surg_proc_glaucoma | Surgical Procedure for Glaucoma |
| OCT Angiography Variables | |
| surg_proc_catextract | Surgical Procedure for Cataract Extraction |
| eyeid | Eye Identification |
| eye_imaged | Eye Imaged |
| time_stamp | Exam Date and Time |
| name_last | Last Name |
| name_first | First Name |
| image_area | Image Area |
| image_method | Image Method |
| image_size | Image Size |
| signal_strength | Signal Strength |
| v_diameter | Vessel Diameter (units?) |
| v_area_density | Vessel Area Density (units?) |
| v_skeleton_density | Vessel Skeleton Density (units?) |
| v_perimeter_index | Vessel Perimeter Index (units?) |
| v_complexity_index | Vessel Complexity Index (units?) |
| flow_impair_zone | Flow Impairment Zone (units?) |
| flux | Flux (units?) |
| zeiss_v_area_density | Vessel Area Density (units?), Zeiss Commercial |
| zeiss_flux | Flux (units?), Zeiss Commercial |
Single Eye Participation
Venugopal (2019) et al. ran the analysis including all eyes.1 Then they re-ran the analysis after selecting only a single eye randomly from each individual. They conclude minimal difference in the findings, and they report only the analysis that includes all eyes:
“The entire analysis was repeated considering one eye per subject (one eye was randomly chosen from subjects contributing both eyes for the primary analysis) and all the results were similar to the primary analysis.”1
The following results are for the single eye, chosen at random, from each participant. Bruce did explain a way to calculate ICC’s for all measurements on both eyes of each participant. This will be coded later.
Subsetting Cohorts
Intra-Visit Data
Intra-visit data was restricted to all measurements of the same eye that occurred on the same day. Some people had multiple measurements on multiple days. Their first day was included only.
Inter-Visit Data
Inter-visit data was restricted to the first measurement of the day for each eye. If a patient had their right eye measured:
3 times on Monday
5 times on Wednesday, and
2 times on Friday,
Then they would have three observations in this dataset:
The first measurement from Monday,
The first measurement from Wednesday, and
The first measurement from Friday.
Data Flow Chart
This tallies how many participants, how many eyes, and how many measurements are included in the two datasets.
| dataset | n_participants | n_eyes | n_measurements |
|---|---|---|---|
| Global Data | |||
| Intra-Visit | 88 | 88 | 214 |
| Inter-Visit | 88 | 88 | 241 |
Table 1: Summary Statistics
Intra-Visit OCTA Summary Statistics
Variables in intra-visit dataset
Participants
| None | Glaucoma | p | test | |
|---|---|---|---|---|
| n | 29 | 59 | ||
| Sociodemographic Variables | ||||
| age (mean (SD)) | 52.31 (17.08) | 60.15 (15.12) | 0.031 | |
| Sex = Male (%) | 13 (44.8) | 30 ( 50.8) | 0.761 | |
| Systemic Clinical Variables | ||||
| Hypertension Diagnosis = Yes (%) | 9 (31.0) | 21 ( 35.6) | 0.853 | |
| Diabetes Mellitus Diagnosis = Yes (%) | 2 ( 6.9) | 11 ( 18.6) | 0.254 | |
| Optic Neuropathy = Yes (%) | 1 ( 3.4) | 0 ( 0.0) | 0.715 | |
| Ophthalmic Clinical Variables | ||||
| Glaucoma Diagnosis, Binary = Glaucoma (%) | 0 ( 0.0) | 59 (100.0) | <0.001 | |
| Diabetic Retinopathy Diagnosis, Binary = Yes (%) | 0 ( 0.0) | 4 ( 6.8) | 0.373 | |
| Ophthalmic Surgical Variables | ||||
| Surgical Procedure for Cataract Extraction = Yes (%) | 5 (17.2) | 13 ( 22.0) | 0.808 | |
| Surgical Procedure for Glaucoma (%) | 0.004 | |||
| None | 25 (86.2) | 31 ( 52.5) | ||
| Laser | 4 (13.8) | 8 ( 13.6) | ||
| Minimally Invasive Glaucoma Surgery | 0 ( 0.0) | 1 ( 1.7) | ||
| Filter | 0 ( 0.0) | 19 ( 32.2) | ||
| OCT Angiography Signal Strength | ||||
| signal_strength (mean (SD)) | 9.29 (0.71) | 9.11 (0.75) | 0.278 | |
Measurements
| Normal | Glaucoma | p | test | |
|---|---|---|---|---|
| n | 73 | 141 | ||
| Sociodemographic Variables | ||||
| age (mean (SD)) | 52.04 (17.18) | 59.48 (15.10) | 0.001 | |
| Sex = Male (%) | 34 ( 46.6) | 73 ( 51.8) | 0.564 | |
| Systemic Clinical Variables | ||||
| Hypertension Diagnosis = Yes (%) | 23 ( 31.5) | 50 ( 35.5) | 0.670 | |
| Diabetes Mellitus Diagnosis = Yes (%) | 4 ( 5.5) | 24 ( 17.0) | 0.031 | |
| Optic Neuropathy = Yes (%) | 3 ( 4.1) | 0 ( 0.0) | 0.070 | |
| Diabetic Retinopathy Diagnosis | ||||
| Diabetic Retinopathy Diagnosis, Ordinal (%) | NaN | |||
| None | 73 (100.0) | 132 ( 93.6) | ||
| Mild Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 5 ( 3.5) | ||
| Moderate Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 2 ( 1.4) | ||
| Severe Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 2 ( 1.4) | ||
| Proliferative Diabetic Retinopathy | 0 ( 0.0) | 0 ( 0.0) | ||
| Glaucoma Suspect Diagnosis | ||||
| Glaucoma Suspect Diagnosis (%) | <0.001 | |||
| None | 3 ( 4.1) | 141 (100.0) | ||
| Mild | 25 ( 34.2) | 0 ( 0.0) | ||
| Moderate | 22 ( 30.1) | 0 ( 0.0) | ||
| Severe | 23 ( 31.5) | 0 ( 0.0) | ||
| Glaucoma Diagnosis, Ordinal | ||||
| Glaucoma Diagnosis, Ordinal (%) | <0.001 | |||
| None | 73 (100.0) | 0 ( 0.0) | ||
| Mild | 0 ( 0.0) | 53 ( 37.6) | ||
| Moderate | 0 ( 0.0) | 38 ( 27.0) | ||
| Severe | 0 ( 0.0) | 50 ( 35.5) | ||
| Surgical Procedure for Glaucoma | ||||
| Surgical Procedure for Glaucoma (%) | <0.001 | |||
| None | 63 ( 86.3) | 76 ( 53.9) | ||
| Laser | 10 ( 13.7) | 18 ( 12.8) | ||
| Minimally Invasive Glaucoma Surgery | 0 ( 0.0) | 3 ( 2.1) | ||
| Filter | 0 ( 0.0) | 44 ( 31.2) | ||
| Surgical Procedure for Cataract Extraction | ||||
| Surgical Procedure for Cataract Extraction = Yes (%) | 12 ( 16.4) | 29 ( 20.6) | 0.586 | |
| Image Specifications | ||||
| Eye Imaged = Right (%) | 39 ( 53.4) | 73 ( 51.8) | 0.932 | |
| Image Area = ONH (%) | 73 (100.0) | 141 (100.0) | NA | |
| Image Method = Angiography (%) | 73 (100.0) | 141 (100.0) | NA | |
| Image Size = 6x6 (%) | 73 (100.0) | 141 (100.0) | NA | |
| OCT Angiography Signal Strength | ||||
| Signal Strength (%) | 0.320 | |||
| 7 | 0 ( 0.0) | 4 ( 2.8) | ||
| 8 | 14 ( 19.2) | 31 ( 22.0) | ||
| 9 | 23 ( 31.5) | 50 ( 35.5) | ||
| 10 | 36 ( 49.3) | 56 ( 39.7) | ||
| OCT Angiography Variables | ||||
| v_diameter (mean (SD)) | 25.81 (0.68) | 27.22 (1.62) | <0.001 | |
| v_area_density (mean (SD)) | 0.36 (0.05) | 0.29 (0.07) | <0.001 | |
| zeiss_v_area_density (mean (SD)) | 0.43 (0.03) | 0.40 (0.04) | <0.001 | |
| v_skeleton_density (mean (SD)) | 0.14 (0.02) | 0.11 (0.03) | <0.001 | |
| v_perimeter_index (mean (SD)) | 0.31 (0.04) | 0.25 (0.06) | <0.001 | |
| v_complexity_index (mean (SD)) | 7881.45 (1056.61) | 6185.71 (1658.25) | <0.001 | |
| flow_impair_zone (mean (SD)) | 9.77 (3.90) | 15.19 (5.74) | <0.001 | |
| flux (mean (SD)) | 0.11 (0.02) | 0.08 (0.03) | <0.001 | |
| zeiss_flux (mean (SD)) | 0.36 (0.03) | 0.32 (0.04) | <0.001 | |
Inter-Visit OCTA Summary Statistics
Variables in inter-visit dataset
Participants
| None | Glaucoma | p | test | |
|---|---|---|---|---|
| n | 29 | 59 | ||
| Sociodemographic Variables | ||||
| age (mean (SD)) | 52.31 (17.08) | 60.15 (15.12) | 0.031 | |
| Sex = Male (%) | 13 (44.8) | 30 ( 50.8) | 0.761 | |
| Systemic Clinical Variables | ||||
| Hypertension Diagnosis = Yes (%) | 9 (31.0) | 21 ( 35.6) | 0.853 | |
| Diabetes Mellitus Diagnosis = Yes (%) | 2 ( 6.9) | 11 ( 18.6) | 0.254 | |
| Optic Neuropathy = Yes (%) | 1 ( 3.4) | 0 ( 0.0) | 0.715 | |
| Ophthalmic Clinical Variables | ||||
| Glaucoma Diagnosis, Binary = Glaucoma (%) | 0 ( 0.0) | 59 (100.0) | <0.001 | |
| Diabetic Retinopathy Diagnosis, Binary = Yes (%) | 0 ( 0.0) | 4 ( 6.8) | 0.373 | |
| Ophthalmic Surgical Variables | ||||
| Surgical Procedure for Cataract Extraction = Yes (%) | 5 (17.2) | 13 ( 22.0) | 0.808 | |
| Surgical Procedure for Glaucoma (%) | 0.004 | |||
| None | 25 (86.2) | 31 ( 52.5) | ||
| Laser | 4 (13.8) | 8 ( 13.6) | ||
| Minimally Invasive Glaucoma Surgery | 0 ( 0.0) | 1 ( 1.7) | ||
| Filter | 0 ( 0.0) | 19 ( 32.2) | ||
| OCT Angiography Signal Strength | ||||
| signal_strength (mean (SD)) | 9.35 (0.65) | 9.09 (0.64) | 0.081 | |
Measurements
| Normal | Glaucoma | p | test | |
|---|---|---|---|---|
| n | 69 | 172 | ||
| Sociodemographic Variables | ||||
| age (mean (SD)) | 53.84 (16.48) | 59.47 (14.94) | 0.011 | |
| Sex = Male (%) | 30 ( 43.5) | 90 ( 52.3) | 0.272 | |
| Systemic Clinical Variables | ||||
| Hypertension Diagnosis = Yes (%) | 23 ( 33.3) | 65 ( 37.8) | 0.616 | |
| Diabetes Mellitus Diagnosis = Yes (%) | 4 ( 5.8) | 33 ( 19.2) | 0.016 | |
| Optic Neuropathy = Yes (%) | 3 ( 4.3) | 0 ( 0.0) | 0.035 | |
| Diabetic Retinopathy Diagnosis | ||||
| Diabetic Retinopathy Diagnosis, Ordinal (%) | NaN | |||
| None | 69 (100.0) | 162 ( 94.2) | ||
| Mild Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 6 ( 3.5) | ||
| Moderate Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 2 ( 1.2) | ||
| Severe Non-Proliferative Diabetic Retinopathy | 0 ( 0.0) | 2 ( 1.2) | ||
| Proliferative Diabetic Retinopathy | 0 ( 0.0) | 0 ( 0.0) | ||
| Glaucoma Suspect Diagnosis | ||||
| Glaucoma Suspect Diagnosis (%) | <0.001 | |||
| None | 3 ( 4.3) | 172 (100.0) | ||
| Mild | 19 ( 27.5) | 0 ( 0.0) | ||
| Moderate | 22 ( 31.9) | 0 ( 0.0) | ||
| Severe | 25 ( 36.2) | 0 ( 0.0) | ||
| Glaucoma Diagnosis, Ordinal | ||||
| Glaucoma Diagnosis, Ordinal (%) | <0.001 | |||
| None | 69 (100.0) | 0 ( 0.0) | ||
| Mild | 0 ( 0.0) | 61 ( 35.5) | ||
| Moderate | 0 ( 0.0) | 43 ( 25.0) | ||
| Severe | 0 ( 0.0) | 68 ( 39.5) | ||
| Surgical Procedure for Glaucoma | ||||
| Surgical Procedure for Glaucoma (%) | <0.001 | |||
| None | 60 ( 87.0) | 82 ( 47.7) | ||
| Laser | 9 ( 13.0) | 23 ( 13.4) | ||
| Minimally Invasive Glaucoma Surgery | 0 ( 0.0) | 2 ( 1.2) | ||
| Filter | 0 ( 0.0) | 65 ( 37.8) | ||
| Surgical Procedure for Cataract Extraction | ||||
| Surgical Procedure for Cataract Extraction = Yes (%) | 14 ( 20.3) | 39 ( 22.7) | 0.817 | |
| Image Specifications | ||||
| Eye Imaged = Right (%) | 36 ( 52.2) | 88 ( 51.2) | 1.000 | |
| Image Area = ONH (%) | 69 (100.0) | 172 (100.0) | NA | |
| Image Method = Angiography (%) | 69 (100.0) | 172 (100.0) | NA | |
| Image Size = 6x6 (%) | 69 (100.0) | 172 (100.0) | NA | |
| OCT Angiography Signal Strength | ||||
| Signal Strength (%) | 0.348 | |||
| 7 | 1 ( 1.4) | 7 ( 4.1) | ||
| 8 | 12 ( 17.4) | 36 ( 20.9) | ||
| 9 | 22 ( 31.9) | 64 ( 37.2) | ||
| 10 | 34 ( 49.3) | 65 ( 37.8) | ||
| OCT Angiography Variables | ||||
| v_diameter (mean (SD)) | 25.87 (0.65) | 27.32 (1.59) | <0.001 | |
| v_area_density (mean (SD)) | 0.36 (0.05) | 0.28 (0.07) | <0.001 | |
| zeiss_v_area_density (mean (SD)) | 0.43 (0.02) | 0.39 (0.04) | <0.001 | |
| v_skeleton_density (mean (SD)) | 0.14 (0.02) | 0.10 (0.03) | <0.001 | |
| v_perimeter_index (mean (SD)) | 0.31 (0.04) | 0.24 (0.06) | <0.001 | |
| v_complexity_index (mean (SD)) | 7790.84 (1119.48) | 5991.62 (1557.79) | <0.001 | |
| flow_impair_zone (mean (SD)) | 9.98 (4.23) | 15.90 (5.61) | <0.001 | |
| flux (mean (SD)) | 0.11 (0.02) | 0.08 (0.02) | <0.001 | |
| zeiss_flux (mean (SD)) | 0.35 (0.03) | 0.32 (0.03) | <0.001 | |
Exploratory Data Analysis
Scatter Plots
Scatter plots of OCTA variables by Subject ID for about 10 individuals randomly sampled. These are separated by intra- and inter-visit datasets, and are useful for understanding the data better.
Vessel Area Density
Vessel Area Density, Zeiss
Vessel Diameter
Vessel Skeleton Density
Vessel Perimeter Index
Vessel Complexity Index
Flow Impairment Zone
Flux
Flux, Zeiss
Spaghetti Plots
Spaghetti plots of OCTA variables for about 10 individuals randomly sampled.
Vessel Area Density
Vessel Area Density, Zeiss
Vessel Diameter
Vessel Skeleton Density
Vessel Perimeter Index
Vessel Complexity Index
Flow Impairment Zone
Flux
Flux, Zeiss
Bland-Altman Plots
Coding of Bland-Altman (BA) plots is explained in this blog post.
Intra-Visit Bland-Altman Plots
Intra-Visit Bland-Altman Plots of OCTA Variables
Vessel Area Density
Vessel Area Density, Zeiss
Vessel Diameter
Vessel Skeleton Density
Vessel Perimeter Index
Vessel Complexity Index
Flow Impairment Zone
Flux
Flux, Zeiss
Outliers
Assess some of the observations outside of expected limits.
| eyeid | ptid | female | eye_imaged | image_area | image_size | image_method | exam_date | time_stamp | v_diameter | v_area_density | v_skeleton_density | v_perimeter_index | v_complexity_index | flow_impair_zone | flux | signal_strength | zeiss_v_area_density | zeiss_flux | age | htn_dx | dm_dx | nro_dx | dr_dx | glc_sus_dx | glc_dx | surg_proc_glaucoma | surg_proc_catextract | glc_dx_bin | dr_dx_bin | n_msr_day | order_obs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 27 | 1787031 | Female | Right | ONH | 6x6 | Angiography | 2019-04-10 | 2019-04-10 11:47:25 | 26.79852 | 0.2269611 | 0.08469167 | 0.1965333 | 4875.438 | 19.4988 | 0.0609257 | 9 | 0.386 | 0.286 | 52 | No | No | No | None | Severe | None | None | Yes | Normal | No | 2 | 1 |
| 27 | 1787031 | Female | Right | ONH | 6x6 | Angiography | 2019-04-10 | 2019-04-10 11:47:52 | 26.37865 | 0.3541333 | 0.13425000 | 0.3041139 | 7481.666 | 10.8069 | 0.1007582 | 8 | 0.378 | 0.280 | 52 | No | No | No | None | Severe | None | None | Yes | Normal | No | 2 | 2 |
Inter-Visit Bland-Altman Plots
Inter-Visit Bland-Altman Plots of OCTA Variables
Vessel Area Density
Vessel Area Density, Zeiss
Vessel Diameter
Vessel Skeleton Density
Vessel Perimeter Index
Vessel Complexity Index
Flow Impairment Zone
Flux
Flux, Zeiss
Outliers
Assess some of the observations outside of expected limits.
| eyeid | ptid | female | eye_imaged | image_area | image_size | image_method | exam_date | time_stamp | v_diameter | v_area_density | v_skeleton_density | v_perimeter_index | v_complexity_index | flow_impair_zone | flux | signal_strength | zeiss_v_area_density | zeiss_flux | age | htn_dx | dm_dx | nro_dx | dr_dx | glc_sus_dx | glc_dx | surg_proc_glaucoma | surg_proc_catextract | glc_dx_bin | dr_dx_bin | order_obs | day_one | diff_date | ind_n_day | n_exams |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 27 | 1787031 | Female | Right | ONH | 6x6 | Angiography | 2018-10-10 | 2018-10-10 10:45:55 | 26.00271 | 0.3302417 | 0.12700278 | 0.2890167 | 7246.137 | 12.2366 | 0.09268497 | 8 | 0.380 | 0.288 | 52 | No | No | No | None | Severe | None | None | Yes | Normal | No | 1 | 2018-10-10 | 0 days | 1 | 3 |
| 27 | 1787031 | Female | Right | ONH | 6x6 | Angiography | 2019-04-10 | 2019-04-10 11:47:25 | 26.79852 | 0.2269611 | 0.08469167 | 0.1965333 | 4875.438 | 19.4988 | 0.06092570 | 9 | 0.386 | 0.286 | 52 | No | No | No | None | Severe | None | None | Yes | Normal | No | 2 | 2018-10-10 | 182 days | 1 | 3 |
| 27 | 1787031 | Female | Right | ONH | 6x6 | Angiography | 2019-05-08 | 2019-05-08 11:09:31 | 26.18709 | 0.3000750 | 0.11458889 | 0.2632389 | 6615.504 | 14.1551 | 0.08483724 | 7 | 0.380 | 0.278 | 52 | No | No | No | None | Severe | None | None | Yes | Normal | No | 3 | 2018-10-10 | 210 days | 1 | 3 |
Intra-Visit Repeatability
Intra-Visit Repeatability of OCTA Variables
Calculate the within-subject standard deviation \((S_w)\), the within-subject coefficient of repeatability \((CR_w)\), and the within-subject coefficient of variation \((CV_w)\) for a given measurement variable \(x\).
Table 2: Repeatability
Table 2 shows repeatability estimates of vessel density measurements.
This table was modeled after Table 2 in Venugopal (2018).2
Intra-Visit Repeatability
Normal
| OCTA Variable | n_measure | µ | Sw | Sw 95% CL | CRw | CRw 95% CL | CVw | CVw 95% CL |
|---|---|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 73 | 25.805 | 0.214 | (0.200, 0.228) | 0.593 | (0.555, 0.631) | 0.830 | (0.776, 0.883) |
| Vessel Area Density (units?) | 73 | 0.360 | 0.019 | (0.019, 0.019) | 0.053 | (0.052, 0.053) | 5.276 | (5.192, 5.360) |
| Vessel Area Density (units?), Zeiss Commercial | 73 | 0.433 | 0.006 | (0.006, 0.006) | 0.017 | (0.017, 0.017) | 1.446 | (1.443, 1.448) |
| Vessel Skeleton Density (units?) | 73 | 0.140 | 0.008 | (0.008, 0.008) | 0.021 | (0.021, 0.021) | 5.512 | (5.480, 5.545) |
| Vessel Perimeter Index (units?) | 73 | 0.315 | 0.016 | (0.016, 0.016) | 0.045 | (0.044, 0.046) | 5.162 | (5.093, 5.231) |
| Vessel Complexity Index (units?) | 73 | 7881.455 | 400.198 | (-126083.230, 126883.626) | 1109.293 | (-349484.883, 351703.469) | 5.078 | (-1599.746, 1609.901) |
| Flow Impairment Zone (units?) | 73 | 9.766 | 1.551 | (0.136, 2.966) | 4.300 | (0.377, 8.222) | 15.884 | (1.393, 30.376) |
| Flux (units?) | 73 | 0.112 | 0.008 | (0.008, 0.008) | 0.022 | (0.022, 0.022) | 7.019 | (6.988, 7.050) |
| Flux (units?), Zeiss Commercial | 73 | 0.357 | 0.006 | (0.006, 0.006) | 0.017 | (0.017, 0.017) | 1.743 | (1.740, 1.745) |
Glaucoma
| OCTA Variable | n_measure | µ | Sw | Sw 95% CL | CRw | CRw 95% CL | CVw | CVw 95% CL |
|---|---|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 141 | 27.215 | 0.328 | (0.308, 0.349) | 0.910 | (0.853, 0.968) | 1.207 | (1.130, 1.283) |
| Vessel Area Density (units?) | 141 | 0.290 | 0.012 | (0.012, 0.012) | 0.033 | (0.033, 0.033) | 4.086 | (4.077, 4.094) |
| Vessel Area Density (units?), Zeiss Commercial | 141 | 0.398 | 0.007 | (0.007, 0.007) | 0.019 | (0.019, 0.019) | 1.763 | (1.759, 1.766) |
| Vessel Skeleton Density (units?) | 141 | 0.108 | 0.005 | (0.005, 0.005) | 0.014 | (0.014, 0.014) | 4.649 | (4.645, 4.654) |
| Vessel Perimeter Index (units?) | 141 | 0.250 | 0.010 | (0.010, 0.010) | 0.029 | (0.029, 0.029) | 4.140 | (4.133, 4.148) |
| Vessel Complexity Index (units?) | 141 | 6185.715 | 261.070 | (-11409.478, 11931.619) | 723.650 | (-31625.459, 33072.759) | 4.221 | (-184.449, 192.890) |
| Flow Impairment Zone (units?) | 141 | 15.192 | 1.291 | (1.000, 1.582) | 3.579 | (2.772, 4.385) | 8.498 | (6.583, 10.413) |
| Flux (units?) | 141 | 0.083 | 0.004 | (0.004, 0.004) | 0.012 | (0.012, 0.012) | 5.091 | (5.087, 5.095) |
| Flux (units?), Zeiss Commercial | 141 | 0.324 | 0.007 | (0.007, 0.007) | 0.018 | (0.018, 0.018) | 2.047 | (2.045, 2.050) |
Calculating Repeatability Statistics
Explanations of repeatability statistic calculations
Within-Subject Mean \((\mu_w)\)
\(\mu_w\) is the average of all measurements of one eye for a given individual. The overall mean \((\mu)\) is calculated by taking the mean of \(\mu_w\) across the dataset. The overall mean is included in the final table for reference.
\[ \mu_w = \frac{1}{M}\sum_{i=1}^{M} x_i \]
\[\text{where } M \text{ is the number of measurements per eye,}\]
Within-Subject Standard Deviation \((S_w)\)
Find \(S_w\) by first calculating the variance of the measurements per individual eye. The equation below is general for any number of measurements; in this study there are at least 2 measurements per eye, but sometimes more.
\[ \sigma^2_{measurement} = \frac{1}{M}\sum_{i=1}^{M} (x_i - \mu_w)^2 \]
\[ \text{where } M \text{ is the number of measurements per eye} \]
To calculate \(S_w\), average of the variance of measurements \((\sigma^2_{measurement})\) for all eyes measured, then take the square root.
\[ Sw = \sqrt{\frac{1}{N}\sum_{i=1}^{N}\sigma^2_{measurement,i}} \]
\[ \text{where } N \text{ is the number of eyes measured} \]
Within-Subject Coefficient of Repeatability \((CR_w)\)
The \(CR_w\) provides the uncertainty of repeated measures.
\[ CRw = \sqrt2 * 1.96 * Sw \]
The \(CR_w\) can be interpreted as:
“The difference between two measurements for the same subject is expected to be less than [\(CR_w\)] for 95% of pairs of observations.”3
Within-Subject Coefficient of Variation \((CV_w)\)
\[ CVw = 100 *\frac{Sw}{\mu} \]
Intraclass Correlation Coefficient, Unadjusted
Intraclass Correlation Coefficient (ICC) is scored using the ICCest() function from the ICC package. This function “estimates the ICC and confidence intervals using the variance components from a one-way ANOVA.”
This is a preliminary result, as we want to calculate the ICC after adjusting for participant, eye (left or right), signal strength, and clinical and other variables. This will be done later in the mixed effects regression models.
Intra-Visit ICC, Unadjusted
Normal
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 29 | 0.900 | (0.818, 0.949) | 2.514 | 0.047 | 0.422 |
| Vessel Area Density (units?) | 29 | 0.858 | (0.746, 0.927) | 2.514 | 0.000 | 0.002 |
| Vessel Area Density (units?), Zeiss Commercial | 29 | 0.939 | (0.886, 0.969) | 2.514 | 0.000 | 0.001 |
| Vessel Skeleton Density (units?) | 29 | 0.880 | (0.783, 0.938) | 2.514 | 0.000 | 0.000 |
| Vessel Perimeter Index (units?) | 29 | 0.866 | (0.761, 0.931) | 2.514 | 0.000 | 0.002 |
| Vessel Complexity Index (units?) | 29 | 0.874 | (0.773, 0.935) | 2.514 | 143516.758 | 995244.605 |
| Flow Impairment Zone (units?) | 29 | 0.857 | (0.746, 0.927) | 2.514 | 2.214 | 13.316 |
| Flux (units?) | 29 | 0.864 | (0.756, 0.930) | 2.514 | 0.000 | 0.000 |
| Flux (units?), Zeiss Commercial | 29 | 0.954 | (0.914, 0.977) | 2.514 | 0.000 | 0.001 |
Glaucoma
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 59 | 0.959 | (0.935, 0.974) | 2.388 | 0.110 | 2.539 |
| Vessel Area Density (units?) | 59 | 0.972 | (0.955, 0.983) | 2.388 | 0.000 | 0.005 |
| Vessel Area Density (units?), Zeiss Commercial | 59 | 0.965 | (0.945, 0.978) | 2.388 | 0.000 | 0.001 |
| Vessel Skeleton Density (units?) | 59 | 0.974 | (0.958, 0.984) | 2.388 | 0.000 | 0.001 |
| Vessel Perimeter Index (units?) | 59 | 0.974 | (0.959, 0.984) | 2.388 | 0.000 | 0.004 |
| Vessel Complexity Index (units?) | 59 | 0.976 | (0.962, 0.985) | 2.388 | 67149.931 | 2711480.499 |
| Flow Impairment Zone (units?) | 59 | 0.950 | (0.921, 0.969) | 2.388 | 1.668 | 31.560 |
| Flux (units?) | 59 | 0.976 | (0.962, 0.985) | 2.388 | 0.000 | 0.001 |
| Flux (units?), Zeiss Commercial | 59 | 0.969 | (0.951, 0.981) | 2.388 | 0.000 | 0.001 |
Inter-Visit ICC, Unadjusted
Inter-Visit ICC Normal
Normal, All
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 29 | 0.662 | (0.442, 0.816) | 2.373 | 0.146 | 0.286 |
| Vessel Area Density (units?) | 29 | 0.799 | (0.645, 0.896) | 2.373 | 0.001 | 0.002 |
| Vessel Area Density (units?), Zeiss Commercial | 29 | 0.898 | (0.811, 0.949) | 2.373 | 0.000 | 0.001 |
| Vessel Skeleton Density (units?) | 29 | 0.790 | (0.632, 0.891) | 2.373 | 0.000 | 0.000 |
| Vessel Perimeter Index (units?) | 29 | 0.794 | (0.638, 0.893) | 2.373 | 0.000 | 0.002 |
| Vessel Complexity Index (units?) | 29 | 0.789 | (0.630, 0.890) | 2.373 | 268745.970 | 1007676.699 |
| Flow Impairment Zone (units?) | 29 | 0.789 | (0.629, 0.890) | 2.373 | 3.849 | 14.366 |
| Flux (units?) | 29 | 0.647 | (0.422, 0.808) | 2.373 | 0.000 | 0.000 |
| Flux (units?), Zeiss Commercial | 29 | 0.875 | (0.771, 0.937) | 2.373 | 0.000 | 0.001 |
Normal, 365 Days
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 15 | 0.592 | (0.168, 0.838) | 2.065 | 0.166 | 0.241 |
| Vessel Area Density (units?) | 15 | 0.798 | (0.521, 0.926) | 2.065 | 0.001 | 0.002 |
| Vessel Area Density (units?), Zeiss Commercial | 15 | 0.883 | (0.704, 0.958) | 2.065 | 0.000 | 0.000 |
| Vessel Skeleton Density (units?) | 15 | 0.801 | (0.529, 0.927) | 2.065 | 0.000 | 0.000 |
| Vessel Perimeter Index (units?) | 15 | 0.789 | (0.503, 0.922) | 2.065 | 0.000 | 0.002 |
| Vessel Complexity Index (units?) | 15 | 0.779 | (0.484, 0.918) | 2.065 | 294366.382 | 1035373.378 |
| Flow Impairment Zone (units?) | 15 | 0.840 | (0.607, 0.942) | 2.065 | 3.557 | 18.628 |
| Flux (units?) | 15 | 0.856 | (0.643, 0.948) | 2.065 | 0.000 | 0.000 |
| Flux (units?), Zeiss Commercial | 15 | 0.935 | (0.828, 0.977) | 2.065 | 0.000 | 0.001 |
Normal, 180 Days
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 6 | 0.757 | (0.093, 0.961) | 2 | 0.106 | 0.329 |
| Vessel Area Density (units?) | 6 | 0.940 | (0.685, 0.991) | 2 | 0.000 | 0.002 |
| Vessel Area Density (units?), Zeiss Commercial | 6 | 0.844 | (0.328, 0.976) | 2 | 0.000 | 0.000 |
| Vessel Skeleton Density (units?) | 6 | 0.947 | (0.722, 0.992) | 2 | 0.000 | 0.000 |
| Vessel Perimeter Index (units?) | 6 | 0.940 | (0.686, 0.991) | 2 | 0.000 | 0.002 |
| Vessel Complexity Index (units?) | 6 | 0.939 | (0.681, 0.991) | 2 | 65655.098 | 1004241.589 |
| Flow Impairment Zone (units?) | 6 | 0.988 | (0.929, 0.998) | 2 | 0.274 | 22.164 |
| Flux (units?) | 6 | 0.861 | (0.383, 0.979) | 2 | 0.000 | 0.000 |
| Flux (units?), Zeiss Commercial | 6 | 0.836 | (0.303, 0.975) | 2 | 0.000 | 0.001 |
Inter-Visit ICC Glaucoma
Glaucoma
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 59 | 0.924 | (0.886, 0.952) | 2.908 | 0.193 | 2.356 |
| Vessel Area Density (units?) | 59 | 0.921 | (0.881, 0.950) | 2.908 | 0.000 | 0.004 |
| Vessel Area Density (units?), Zeiss Commercial | 59 | 0.943 | (0.914, 0.964) | 2.908 | 0.000 | 0.001 |
| Vessel Skeleton Density (units?) | 59 | 0.925 | (0.887, 0.952) | 2.908 | 0.000 | 0.001 |
| Vessel Perimeter Index (units?) | 59 | 0.926 | (0.889, 0.953) | 2.908 | 0.000 | 0.003 |
| Vessel Complexity Index (units?) | 59 | 0.930 | (0.895, 0.956) | 2.908 | 171411.493 | 2286561.423 |
| Flow Impairment Zone (units?) | 59 | 0.856 | (0.789, 0.907) | 2.908 | 4.574 | 27.256 |
| Flux (units?) | 59 | 0.924 | (0.885, 0.951) | 2.908 | 0.000 | 0.001 |
| Flux (units?), Zeiss Commercial | 59 | 0.917 | (0.876, 0.947) | 2.908 | 0.000 | 0.001 |
Glaucoma, 365 Days
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 48 | 0.924 | (0.878, 0.955) | 2.597 | 0.226 | 2.748 |
| Vessel Area Density (units?) | 48 | 0.938 | (0.900, 0.963) | 2.597 | 0.000 | 0.005 |
| Vessel Area Density (units?), Zeiss Commercial | 48 | 0.946 | (0.913, 0.968) | 2.597 | 0.000 | 0.001 |
| Vessel Skeleton Density (units?) | 48 | 0.942 | (0.907, 0.966) | 2.597 | 0.000 | 0.001 |
| Vessel Perimeter Index (units?) | 48 | 0.942 | (0.907, 0.966) | 2.597 | 0.000 | 0.004 |
| Vessel Complexity Index (units?) | 48 | 0.945 | (0.912, 0.967) | 2.597 | 154027.191 | 2659432.004 |
| Flow Impairment Zone (units?) | 48 | 0.877 | (0.807, 0.926) | 2.597 | 4.416 | 31.554 |
| Flux (units?) | 48 | 0.940 | (0.903, 0.964) | 2.597 | 0.000 | 0.001 |
| Flux (units?), Zeiss Commercial | 48 | 0.928 | (0.884, 0.957) | 2.597 | 0.000 | 0.001 |
Glaucoma, 180 Days
| OCTA Variable | n_people | ICC | ICC 95% CL | k | varw | vara |
|---|---|---|---|---|---|---|
| Vessel Diameter (units?) | 37 | 0.928 | (0.876, 0.961) | 2.454 | 0.248 | 3.218 |
| Vessel Area Density (units?) | 37 | 0.946 | (0.906, 0.971) | 2.454 | 0.000 | 0.006 |
| Vessel Area Density (units?), Zeiss Commercial | 37 | 0.948 | (0.910, 0.972) | 2.454 | 0.000 | 0.002 |
| Vessel Skeleton Density (units?) | 37 | 0.950 | (0.912, 0.973) | 2.454 | 0.000 | 0.001 |
| Vessel Perimeter Index (units?) | 37 | 0.949 | (0.910, 0.972) | 2.454 | 0.000 | 0.005 |
| Vessel Complexity Index (units?) | 37 | 0.950 | (0.913, 0.973) | 2.454 | 165298.655 | 3171823.168 |
| Flow Impairment Zone (units?) | 37 | 0.886 | (0.805, 0.937) | 2.454 | 4.908 | 38.051 |
| Flux (units?) | 37 | 0.948 | (0.909, 0.972) | 2.454 | 0.000 | 0.001 |
| Flux (units?), Zeiss Commercial | 37 | 0.936 | (0.889, 0.965) | 2.454 | 0.000 | 0.002 |
Calculating ICC Statistic
\[ \text{ICC} = \frac{\text{between}}{\text{between} + \text{within}}\]
“In the output for the random effects model ‘SID’ is the estimated between-group variance, which in the ICCest output is called ‘vara’ (variance among groups). The within-group variance is ‘Residual’ in the random effects model output, while in ICCest it’s called ‘varw.’ You can see that the values for those two variances match perfectly in the two different outputs. The ICC is defined as between/(between+within). That calculation is not part of the random effects model output, but it is part of the ICCest output, along with confidence intervals for the ICC estimate. The ICC are really high for intravisit (around 97%) and a little lower for intervisit (about 94%). If we were to calculate these ICC for glaucoma and controls separately I think they would be significantly lower because of the reduced between-group variability.” - BB 9/4/2019
ARVO Abstract Tables
Table 1: Participants
There were:
73 measurements of 29 patients without glaucoma
141 measurements of 59 patients with glaucoma
| None | Glaucoma | p | test | |
|---|---|---|---|---|
| n | 29 | 59 | ||
| Sociodemographic Variables | ||||
| age (mean (SD)) | 52.31 (17.08) | 60.15 (15.12) | 0.031 | |
| Sex = Male (%) | 13 (44.8) | 30 ( 50.8) | 0.761 | |
| Systemic Clinical Variables | ||||
| Hypertension Diagnosis = Yes (%) | 9 (31.0) | 21 ( 35.6) | 0.853 | |
| Diabetes Mellitus Diagnosis = Yes (%) | 2 ( 6.9) | 11 ( 18.6) | 0.254 | |
| Optic Neuropathy = Yes (%) | 1 ( 3.4) | 0 ( 0.0) | 0.715 | |
| Ophthalmic Clinical Variables | ||||
| Glaucoma Diagnosis, Binary = Glaucoma (%) | 0 ( 0.0) | 59 (100.0) | <0.001 | |
| Diabetic Retinopathy Diagnosis, Binary = Yes (%) | 0 ( 0.0) | 4 ( 6.8) | 0.373 | |
| Ophthalmic Surgical Variables | ||||
| Surgical Procedure for Cataract Extraction = Yes (%) | 5 (17.2) | 13 ( 22.0) | 0.808 | |
| Surgical Procedure for Glaucoma (%) | 0.004 | |||
| None | 25 (86.2) | 31 ( 52.5) | ||
| Laser | 4 (13.8) | 8 ( 13.6) | ||
| Minimally Invasive Glaucoma Surgery | 0 ( 0.0) | 1 ( 1.7) | ||
| Filter | 0 ( 0.0) | 19 ( 32.2) | ||
| OCT Angiography Signal Strength | ||||
| signal_strength (mean (SD)) | 9.29 (0.71) | 9.11 (0.75) | 0.278 | |
Table 2: Repeatability Statistics
| var_name_normal | µ_normal | µ_glc | ICC_pretty_normal | ICC_pretty_cl_normal | ICC_pretty_glc | ICC_pretty_cl_glc | Sw_pretty_normal | Sw_pretty_cl_normal | Sw_pretty_glc | Sw_pretty_cl_glc | CRw_pretty_normal | CRw_pretty_cl_normal | CRw_pretty_glc | CRw_pretty_cl_glc | CVw_pretty_normal | CVw_pretty_cl_normal | CVw_pretty_glc | CVw_pretty_cl_glc |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Vessel Area Density (units?) | 0.360 | 0.290 | 0.858 | (0.746, 0.927) | 0.972 | (0.955, 0.983) | 0.019 | (0.019, 0.019) | 0.012 | (0.012, 0.012) | 0.053 | (0.052, 0.053) | 0.033 | (0.033, 0.033) | 5.276 | (5.192, 5.360) | 4.086 | (4.077, 4.094) |
| Vessel Area Density (units?), Zeiss Commercial | 0.433 | 0.398 | 0.939 | (0.886, 0.969) | 0.965 | (0.945, 0.978) | 0.006 | (0.006, 0.006) | 0.007 | (0.007, 0.007) | 0.017 | (0.017, 0.017) | 0.019 | (0.019, 0.019) | 1.446 | (1.443, 1.448) | 1.763 | (1.759, 1.766) |
| Vessel Skeleton Density (units?) | 0.140 | 0.108 | 0.880 | (0.783, 0.938) | 0.974 | (0.958, 0.984) | 0.008 | (0.008, 0.008) | 0.005 | (0.005, 0.005) | 0.021 | (0.021, 0.021) | 0.014 | (0.014, 0.014) | 5.512 | (5.480, 5.545) | 4.649 | (4.645, 4.654) |
| Flux (units?) | 0.112 | 0.083 | 0.864 | (0.756, 0.930) | 0.976 | (0.962, 0.985) | 0.008 | (0.008, 0.008) | 0.004 | (0.004, 0.004) | 0.022 | (0.022, 0.022) | 0.012 | (0.012, 0.012) | 7.019 | (6.988, 7.050) | 5.091 | (5.087, 5.095) |
| Flux (units?), Zeiss Commercial | 0.357 | 0.324 | 0.954 | (0.914, 0.977) | 0.969 | (0.951, 0.981) | 0.006 | (0.006, 0.006) | 0.007 | (0.007, 0.007) | 0.017 | (0.017, 0.017) | 0.018 | (0.018, 0.018) | 1.743 | (1.740, 1.745) | 2.047 | (2.045, 2.050) |
Mixed Effects Regression Models
Model Fitting
“For the markdown file let’s just spit out all the output from the random effects model and the iccest command so we can show it to Grace. Eventually we can put the ICC in the table with the other repeatability measures you have already tabled but I don’t think we need it for the meeting.” - BB 9/4/2019
## Linear mixed model fit by REML ['lmerMod']
## Formula: v_area_density ~ (1 | ptid)
## Data: data_octa_global_intra
##
## REML criterion at convergence: -848.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -4.5112 -0.4821 0.0322 0.4694 4.4542
##
## Random effects:
## Groups Name Variance Std.Dev.
## ptid (Intercept) 0.0048661 0.06976
## Residual 0.0002012 0.01418
## Number of obs: 214, groups: ptid, 88
##
## Fixed effects:
## Estimate Std. Error t value
## (Intercept) 0.310503 0.007502 41.39“I also added a model to test whether SS influences reliability. The reference level is ‘10’ now, which is what we want, and you can see that SS is highly associated with VAD, meaning it affects reliability.” - BB 9/4/2019
## Linear mixed model fit by REML ['lmerMod']
## Formula: v_area_density ~ signal_strength + (1 | ptid)
## Data: data_octa_global_intra
##
## REML criterion at convergence: -865.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.2739 -0.3948 0.0172 0.4053 5.2392
##
## Random effects:
## Groups Name Variance Std.Dev.
## ptid (Intercept) 0.0043478 0.06594
## Residual 0.0001765 0.01328
## Number of obs: 214, groups: ptid, 88
##
## Fixed effects:
## Estimate Std. Error t value
## (Intercept) 0.196002 0.022391 8.753
## signal_strength 0.012486 0.002316 5.391
##
## Correlation of Fixed Effects:
## (Intr)
## sgnl_strngt -0.949References
1. Venugopal JP, Rao HL, Weinreb RN, et al. Repeatability and comparability of peripapillary vessel density measurements of high-density and non-high-density optical coherence tomography angiography scans in normal and glaucoma eyes. The British Journal of Ophthalmology 2019;103:949–954.
2. Venugopal JP, Rao HL, Weinreb RN, et al. Repeatability of vessel density measurements of optical coherence tomography angiography in normal and glaucoma eyes. The British Journal of Ophthalmology 2018;102:352–357.
3. Bland JM, Altman DG. Statistics Notes: Measurement error. BMJ : British Medical Journal 1996;313:744.